1. Field of the Invention
The present invention relates to technology for suppressing deterioration in image quality due to a foreign substance adhering to the surface of an optical low-pass filter or the like disposed in front of an image sensor in an image sensing apparatus that uses an image sensor such as a CCD sensor or a CMOS sensor, and particularly relates to technology for generating foreign substance information.
2. Description of the Related Art
With an interchangeable-lens digital camera, dirt or the like floating in the air may enter into the camera body when the lens is removed from the camera body. In addition, various mechanical units that operate mechanically, such as shutter mechanisms and the like, are disposed inside the camera; thus, there are also cases in which, for example, metal fragments and other such dust is created through the operation of those mechanical units.
When such foreign substances such as dust or dirt adhere to the surface of an optical element, such as an optical low-pass filter, disposed in front of the image sensor of which the sensing unit of the digital camera is configured, the foreign substance appears in shot images as a shadow, which degrades the quality of the shot image.
In order to solve such a problem, a method for correcting the pixel in which the foreign substance appears using signals from pixels surrounding that pixel or the like is conceivable. As such pixel-correction technology, Japanese Patent Laid-Open No. H6-105241, for example, proposes an image defect correction method for correcting pixel defects in an image sensor. Also, Japanese Patent Laid-Open No. 2004-242158 proposes changing the extensions of image files shot in a dust acquisition mode to be different from the extensions of normal images in order to simplify settings for information on the position of the pixel defect. As a result, dust information is automatically determined by a PC (personal computer), and the image to be corrected is then corrected using that information.
In recent years, technology for encoding moving image information at a high compression rate and high quality has been proposed and become widespread in recent years, with the aim of handling such moving image information as digital data for storage and transmission.
The Motion JPEG system encodes data by applying still image encoding (for example, JPEG (Joint Photographic Coding Experts Group) encoding) to each frame. Although JPEG encoding is an encoding system used primarily for still images, products that apply such encoding to moving images through high-speed processing have also emerged.
Meanwhile, the H.264 (MPEG 4-Part10 AVC) is an encoding system that aims at even higher compression rates and image quality. Compared to conventional encoding systems such as MPEG 2 and MPEG 4, it is known that although the H.264 requires greater amounts of computations for encoding and decoding, the H.264 is also capable of realizing even higher encoding efficiency (see ISO/IEC 14496-10, “Advanced Video”).
Compact digital cameras capable of recording moving images using such encoding systems have also been developed and commercialized, and users can easily view images using these apparatuses, or using personal computers, DVD players, or the like.
Under such circumstances, in recent years, there is greater demand for recording moving images at higher resolutions and with more pixels not only with compact digital cameras, but with interchangeable-lens digital cameras as well. However, as already described, since dust adheres to the surface of the image sensor in interchangeable-lens digital cameras due to various factors, there is the possibility that, if a moving image is recorded in that state, dust will always be displayed at the same position while playing back the moving image.
In conventional dust removal methods for interchangeable-lens digital cameras, foreign substance information necessary for removing dust (for example, information regarding the position and the size of the dust) and image data is recorded, the image data is imported into a personal computer or the like later, and the dust is then removed through image processing. Since conventional foreign substance information is applied to image data including still images, such information is acquired through the same shooting operation as that of still image shooting; thus, the information is not foreign substance information generated with moving image recording taken into account.
However, unlike still image shooting, when recording a moving image, control may be performed so as to prioritize processing speed for shooting and recording processing in order to achieve the prescribed frame rate, the maximum moving image recording time, and so on.
For example, with still image shooting, the image data read out from the image sensor is always read out at the maximum size, even when recording images of different sizes. Then, an image processing circuit performs reduction processing on the read-out image data, reducing the data to the size at which the image is to be recorded, after which a recording unit records the data.
However, it takes extremely long to perform processing for reading out image data at the maximum size from the image sensor, and it is difficult to read out image data at the maximum size when recording moving images. Thus, control is performed so as to read out image data that has been reduced at the point in time when the image data is read out from image sensor, in order to reduce the processing time.
Furthermore, the method of the reduction processing is determined when the image processing circuit performs the reduction processing on the image data that was read out, taking the processing time into account.
As described above, the image sensor and the image processing circuit are controlled in a different manner when recording moving images than when shooting still images; thus, the number of dust particles that remain in the recorded data, the size thereof, the shape thereof, and so on are different than when shooting still images.